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Abstract 


This  paper  unifies  the  development  of  the  cutting  plane  algorithm  for 
mathematical  programs  and  variational  inequalities  by  providing  one 
common  framework  for  establishing  convergence.  Strategies  for  generating 
cuts  are  provided  for  cases  in  which  the  algorithm  yields  easy  and  difficult 
subproblems.  When  the  subproblem  is  easy  to  solve,  a  line  search  is  added 
and  a  deep  cut  is  selected  to  accelerate  the  algorithm.  On  the  other  hand, 
when  the  subproblem  is  difficu.'t  to  solve,  the  problem  is  only  solved 
approximately  during  the  early  iterations.  This  corresponds  to  generating 
cuts  which  are  nontangential  to  the  underlying  objective  function.  Moreover, 
in  the  case  of  variational  inequalities,  it  is  shown  further  that  the  subproblem 
can  be  eliminated  entirely  from  the  algorithmic  steps,  thereby  making  the 
resulting  algorithm  especially  advantageous.. 


Aoooaalon  For 

y 

NTIS  QRAM 

DTIC  TAB 

□ 

Unannounced 

□ 

Justification — 

i  By - - - — - - 

Watrtbution/ _ _ 

Availability  CcdPA 


j"  Avail  aud/cr 
iDlat  Spuclal 


I.  INTRODUCTION 


Many  have  studied  the  cutting  plane  approach  for  mathematical 
programs  for  some  time.  Cheney  and  Goldstein  (1959)  and  Kelly  (1960)  first 
proposed  it  for  convex  programs.  Dantzig-Wolfe  (1960)  (see  also  Dantzig, 
1963)  proposed  the  dual  equivalent  called  generalized  linear  programming  or 
column  generation.  Zangwill  (1969)  provided  new  viewpoints  and  supplied 
a  convergence  proof  based  on  algorithmic  maps.  Magnanti  et  al.  (1976) 
showed  that  generalized  linear  programming  solves  Lagrangian  dual 
problems  even  when  standard  convexity  assumptions  do  not  hold.  As  a 
standard  technique,  the  cutting  plane  approach  often  appears  in  texts  such  as 
Bazaraa  and  Shetty  (1979). 

For  variational  inequalities,  references  on  the  cutting  plane  approach  are 
considerably  less.  Zuhovickii  et  al.  (1969)  (see  also  Auslender,  1976)  were  the 
first  to  apply  it  to  variational  inequalities.  Nguyen  and  Dupuis  (1984) 
proposed  an  acceleration  idea  and  prove  convergence  using  Zangwill's 
algorithmic  maps. 

It  is  well  known  (see,  e.g.,  Auslender,  1976,  Hearn  et  al.,  1984  and  Nguyen 
and  Dupuis,  1984)  that  mathematical  programs  (MPs)  and  variational 
inequalities  (Vis)  are  related  and  possess  many  similar  properties.  In  fact, 
many  algorithms  for  VI  problems  are  based  on  MP  algorithms  (see,  Harker 
and  Pang,  1990).;  In  spite  of  this  relationship,  the  developments  of  the  cutting 
plane  algorithm  appear  different  for  these  two  areas.  However,  one  can  unify 
the  developments  by  addressing  MPs  and  Vis  in  a  common  framework.  In 
this  paper,  we  provide  one  such  framework  with  which  tiie  cutting  plane 
algorithm  can  be  derived  for  both  MPs  and  Vis.  This  framework  is  then  used 
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to  analyze  existing  schemes  for  accelerating  the  cutting  plane  algorithm.  The 
analysis  results  in  a  common  argument  for  proving  convergence  for  all 
acceleration  schemes  regardless  of  the  underlying  problems  and  points  out 
when  each  scheme  might  be  effective. 

For  MPs,  this  paper  focuses  on  problems  derived  from  Benders 
decomposition  and  Lagrangian  duality.  For  Vis,  the  paper  considers  those 
with  finite  dimension.  In  Section  2,  it  is  shown  that  these  problems  can  be 
stated  as  the  following  maximin  problem: 

w*  *  max  min  {f{x)  +  uiKx)} 

ueU  X£X 

where  U  and  X  are  nonempty  and  convex  subsets  of  R'"  and  R”,  respectively, 
f(x)  is  a  continuous  real-valued  function  define  on  X,g{x)  is  a  continuous 
vector-valued  function  mapping  R"  into  R'",  and  w*  is  the  maximin  value  for 
the  problem.  For  convenience,  ug{x)  denotes  the  dot  product  of  vectors  u  and 
g(x).  The  basic  cutting  plane  algorithm  is  stated  in  Section  3  with  a 
convergence  proof.  Section  4  utilizes  the  framework  given  in  Section  3  to 
analyze  existing  strategies  for  accelerating  the  algorithm..  Finally,  Section  5 
concludes  the  paper. 
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2.  INSTANCES  OF  MAXIMIN  OPTIMIZATION 

The  first  two  instances  of  the  maxiirin  problem  were  first  observed  in 
Magnanti  and  Wong  (1981)  and  they  are  stated  here  for  completeness.  The  last 
instance  can  be  found  in  a  slightly  (different  form  in  Hearn  et  al.  (1984)  and 
Nguyen  and  Dupuis  (1984). 

First,  consider  the  following  nonlinear  program  (NLP) 

min  f{x) 
s.t.  ^(x)<0 
X  e  X 

where  f{x),  g{x),  and  X  are  as  previously  defined  with  the  exception  that  X  is 
additionally  assumed  to  be  compact.  Then,  the  Lagrangian  dual  problem  of 
the  NLP  can  be  stated  as 

max  min  {/(.t)  +  w  v(x)} 
uiO  xeX 

where  u  now  represents  the  ducil  vector  for  the  constraints  defined  by  ^(x).: 
This  shows  that  Lagrangian  duality  leads  to  the  desired  maximin  problem. 

When  fix)  =  cx,  g(x)  =  b-Ax,  and  X  =  {x:,  Dx  ^  d,  x  >  0).  The  above  NLP 
reduces  to  the  following  linear  program  (LP): 

min  cx 
s.t.  Ax^b 
Dxeid 

X  >0, 

and  the  Lagrangian  dual  of  this  LP  is 

max  min  {cx  +  uib  •  Ax)} , 
u>0  xeX 
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Letting  f{x)  =  cx  and  g{x)  =  b-Ax  yields  the  desired  maximin  problem. 
However,  an  equivalent  method  of  obtaining  the  Lagrangian  dual  of  the  LP  is 
by  applying  Benders  decomposition  to  the  dual  of  the  LP  which  can  be  stated 
as 

max  ub+vd 
s.t.  uA+vD  ^  c 
u,v^0. 

Partitioning  the  decision  variables  (u,v)  gives 

max  max  {ub+vd:  vD<c  ■  uA}.. 
uiO  vlQ 

By  dualizing  the  inner  maximization,  the  above  maximin  problem  can  be 
written  as 

max  min  {ub  +  {c  -  or 

*€X 

max  min  [cx  +  u(b  -  /lx)}. 

M^O  xeX 

The  last  maximin  problem  is  the  same  as  the  one  obtained  with  Lagrangian 
duality. 

Our  last  instance  of  a  maximin  problem  is  a  variational  inequality  (VI) 
problem  which  can  be  stated  as  follows:  find  a  vector  x*  e  S  such  that 

F(x  *)(u  -  x"*)  >  0  Vu  eS 

where  S  is  a  nonempty,  convex  and  compact  subset  of  R"  and  F(x)  is  a  vector 
function  mapping  R"  into  R".  When  f(x)  is  strongly  monotone  on  S,  i.e.,  3  a  > 
0  such  that 

(F(x)  -  F(u)){x  - 1<)  >  a  I X  - 1/ 1  ^  V  X,  M  e  S, 
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then  the  VI  problem  can  be  stated  as  [see  H.’arn  et  al.  (1984)  and  Nguyen  and 
Dupuis  (1984)] 

max  min  {F(ar)(x  -  «)},  or  max  min  {F(a:)a:  -  mF(x)}. 

«€S  X€S  U€S  X€S 

Letting  f{x)  =  F{x)x  and  g(x)  =  -F(x)  again  yields  the  desired  maximin  problem. 
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3.  THE  CUTTING  PLANE  ALGORITHM 


In  the  basic  cutting  plane  algorithm,  the  optimal  value  of  the  inner 
minimization  of  the  maximin  problem  at  a  given  point  u  is  denoted  as  L(u), 
i.e., 

[S]:  L(u)  •  min{/(x)  +  w^x)} 


Since  L(u)  is  defined  as  the  pointwise  minimum  of  a  set  of  functions  linear  in 
u,  L(u)  must  be  concave.,  To  motivate  the  algorithm,  the  maximin  problem  is 
restated  as 

[P]:  w*  =  max  w 

s.t.  XU  S  +  tig(y/)  Vxi  €  X 
u  e  U  and  w  is  unrestricted.; 

Problem  [P]  has  an  infinite  number  of  constraints  of  the  form 

w  <f(x’)  +  »g(x') 

which  are  generally  referred  to  as  cutting  planes  or  simply  cuts.  To  avoid 
generating  all  the  cuts  apriori,  the  algorithm  initially  solves  an  approximation 
of  problem  [P]  which  contains  only  a  few  cuts  and  obtain,  say  (xu',u’),  as  a 
solution.  To  further  refine  the  approximation  and  hence  obtain  a  more 
accurate  solution,  problem  [S]  is  solved  with  u  =  u'  to  produce  a  solution  x' 
which  defines  a  new  cut 

w  <f{x)  +  ug{x'). 


6 


The  approximation  of  problem  [P]  is  updated  and  resolved  with  the  addition 
of  this  new  cut.  Then,  the  process  is  repeated  until  an  optimal  solution  is 
foimd.  Below  we  formally  state  the  algorithm  for  the  maximin  problem. 


The  Cutting  Plane  (CP)  Algorithm 

Step  0:  Find  a  point  x°  e  X.  Set  ^:  =  1  and  go  to  Step  1. 

Step  1:  Solve  the  {k^ )  master  problem 

max  w 

s.t.  w  S  /(x‘)  +  ugix')  i  =  0, ...,  k-l 
ii  E  U  and  w  unrestricted.; 

Let  (w*,  denote  an  optimal  solution  and  go  to  Step  2., 

Step  2:  Solve  the  {k^  )  subproblem 

-argminl/Cx) 
xex  ^ 

andi.(u‘)./(*V"V4 

Step  3:  If  then  (»*■',  .y^)  solves  the  maximin  problem. 

Otherwise  (i.e.,  >  L(iiM),  replace  k  by  k+l  and  go  to  Step  1.; 


In  Step  1,  the  dual  of  the  k^^  master  problem  is  the  following  linear  pro¬ 
gram 


k-l 

min  I  Ttifix^) 

j-0 

it-1 

s.t.  1  n,  six') 

<  0 

1=0 

it  -1 

I 

=  1 

/=0 

n. 

>  0 

Vi  =0, •••,(/:  -1) 
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The  CP  algorithm  with  the  master  problem  replaced  by  its  dual  as  stated 
above  is  generally  known  as  Dantzig-Wolfe  decomposition,  column  genera¬ 
tion,  or  generalized  linear  programming. 

In  both  Benders  decomposition  and  Lagrangian  dual  of  an  NLP,  the  set  X 
is  either  constructed  or  chosen  to  facilitate  the  solving  of  the  subproblem  in 
Step  2  or,  equivalently,  problem  [S].  In  many  cases,  a  closed  form  solution  or 
efficient  algorithm  exists  for  the  subproblem.  However,  such  is  not  the  case 
for  Vis.  The  choice  for  X  is  restricted  to  be  S,  the  feasible  region  of  the  VI 
problem.  Moreover,  the  objective  function  of  the  subproblem  contains  the 
term  F(x)x  which  is  at  least  quadratic,  unless  F(x)  is  a  constant  vector.. 
Therefore,  the  subproblem,  or  problem  [S],  does  not  generally  admit  an  easy 
solution  and  the  CP  algorithm  as  stated  above  would  be  inefficient,  if  not 
ineffective,  for  Vis.  In  Section  4,  a  technique  to  overcome  this  difficulty  is 
discussed.. 

Step  3  uses  the  fact  that  >  L(u^)  which  is  generally  known  to  follow  from 
Lagrangian  duality..  However,  the  inequality  can  also  be  obtained  by  noting 
that  xi  €  X  for  all  j  and 

L(u^)  “  min{/(x)  +  i/^(x)}  <  minMx^)  +  i<*‘g(x^):0  <  /  <  /c  - 1|  =  w'^.: 

So,  regardless  of  how  f(x)  and  g(x)  are  derived,  the  inequality  holds. 

The  following  two  lemmas  are  essential  for  the  convergence  of  the  CP 
algorithm.  The  proofs  of  both  lemmas  are  similar  to  those  in  Dantzig  (1963) 
and  Magnanti  et  al.  (1976)  with  the  exception  that  all  references  to  Lagrangian 
duality  are  eliminated.  However,  they  are  stated  here  for  completeness  and 
further  reference. 
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Lemma  1:  At  each  iteration  of  the  CP  algorithm 

(i)  w^-'^  >w^  'V  k 

(ii)  w*  y  k 

(iii)  If  L{u^)  S  then  w*  =  Liu'^)  =  w^. 

Proof:  The  first  result  follows  from  the  fact  that  the  k^^  master  problem  has 
more  cuts  than  the  The  second  follows  from  the  fact  that  the  master 

problem  in  Step  1  contains  only  a  finite  subset  of  cuts  in  problem  [P]. 

If  L(u*)  >  we  have 

w*<  xu^ 

<  L(i/)  =  minfflA:)  +  i/g(x)} 

X  c  X 

<  maxmin{/‘(Ar)  +  wg(Ar)}  ^  xv* 

«  eU  X  €X 

where  the  first  inequality  follows  from  (ii),  the  second  is  given  in  (iii),  and  the 
last  inequality  is  because  of  the  maximization  over  the  set  U.  The  above  series 
of  inequalities  implies  that  (iii)  holds.  □ 

Lemma  2:  If  X  is  compact  and  there  exists  a  subset  K  £  (1,  2, ...}  such  that  the 
subsequence  is  convergent,  say  to  the  limit  point,  u”,  then 

limu’*^  =  xo*  = 
keK  ^  ' 

Proof:  Since  (lu*^,  u^)  solves  the  k^  master  problem  and  the  fact  that  constraints 
in  the  master  problem,  with  /  <  k,  are  always  contained  in  the  problem  at 
iteration  we  have  that 

f{x^)  +  u'^g{x’)  >  7'/  2  7y*  foi  j  =  0,l,...,(fc  -  1)  (1) 

where  the  last  inequality  follows  from  Lemma  1.,  Let 
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k£K 

Note  that  w"  exists  because  is  a  monotonically  decreasing  sequence 

which  is  bounded  below  by  iv*.  Taking  the  limit  in  equation  (1)  for  k  £  K,  we 
obtain 

>  a;*”  w*  for;  “  0,1,2,3, ...  (2) 

By  assumption,  g{x)  is  continuous  and  X  is  compact..  There  must  exist  a  posi¬ 
tive  real  number  (i  such  that  <  J3  for  all  x  £  X..  Then, 

-  /(/) .  .rs(/)l =|(„‘ .  4/  ■  (3) 

Consequently,  for  any  given  e  >  0,  there  is  a  ki  £  K  such  that  for  all  /:  £  K  and 
^:  >  ^1,  the  last  term  in  (3)  is  bounded  by  e..  Therefore, 

or  £  >  fix'")  +  -  fix'"]  -  irg(x^}  >  -€.■ 

Examining  the  right  inequality,  we  obtain 

-  /("‘l  +  ^  /(**!  *  "”«(**)  •  £ 

or  I.(u‘)rri/(rV.rx(/). 

from  (2)  and  the  definition  of  w*, 

w*  +e  >  L^u^]  +  £  >  f[x^)  +  >  zu”  >  TO*. 

Since  e  is  arbitrary,  we  can  conclude  that 
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w* 


= 

k^K 


where  the  middle  equality  follows  from  the  continuity  of  L{u).  □ 


To  obtain  a  solution  to  the  maximin  problem,  evaluate  the  function  L(u) 
at  M”  to  produce  x”  ,i.e., 

x”  «=  argmm{/(x)  +  M“g(x)}. 

Then,  it  follows  from  Lemma  2  that  (x”,  1/”“)  solves  the  maximin  problem. 

We  now  obtain  convergence  for  the  three  instances  of  maximin  problems. 
For  Benders  decomposition  of  an  LP,  the  second  condition  in  Lemma  1  will 
hold  after  a  finite  number  of  iterations  because  the  set  X  is  assumed  to  be 
compact  and  can  be  expressed  as  a  convex  combination  of  a  finite  number  of 

extreme  points  (see,  e.g.,  Bazaraa  et  al.,  (1990)]..  For  VI  problems,  both  U  and  X 
are  the  same  as  S  which  is  compact.  Since  w*'  €  S  for  all  k,  must  contain  a 

convergent  subsequence,  thereby  satisfying  the  hypothesis  of  Lemma  2.:  So, 
the  CP  algorithm  converges  for  Vis.  The  convergence  for  the  Lagrangian  dual 
of  an  NLP  follows  from  the  theorem  below. 


Theorem  3:  If  there  exists  an  a:°  €  X  such  that  g{x°)  <  0  then  there  is  a 
converging  subsequence  of  .; 

Proof:  See  Fisher  and  Shapiro  (1974). 

This  theorem  also  points  out  that  the  selection  of  the  initial  solution  x°  in 
Step  0  of  the  CP  algorithm  is  critical  for  the  Lagrangian  dual  case,  for  it  deter¬ 
mines  the  convergence  of  the  algorithm,  Magnanti  et  al,  (1976)  provides  a 
procedure  to  obtain  an  initial  solution  satisfying  the  condition  in  Theorem  3  if 
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one  is  not  readily  available.  This  procedure  is  akin  to  phase  one  of  the  two 
phase  method  in  LP. 
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4.  STRATEGIES  FOR  GENERATING  CUTS 


The  two  main  steps  in  the  CP  algorithm  consist  of  solving  the  master 
problem  and  generating  a  new  cut.  In  this  section,  we  examine  existing 
schemes  for  generating  new  cuts. 

Solving  the  subproblem  in  Step  2  of  the  CP  algorithm  is  one  method  of 
generating  new  cuts.  However,  it  does  not  distinguish  between  easy  and 
difficult  subproblems.  When  the  subproblem  is  easy  to  solve,  it  would  be 
advantageous  to  solve  more  subproblems  in  a  effort  to  obtain  better  cuts,  i.e., 
those  which  may  lead  to  a  reduction  in  the  number  of  master  problems  to  be 
solved.  On  the  other  hand,  when  the  subproblem  is  difficult  to  solve,  it  may 
be  better  to  cheaply  obtain  a  legitimate  cut,  perhaps  not  necessarily  tangential 
to  L(u).:  The  two  subsections  below  discuss  these  two  schemes  in  detail. 

4.1:  Easy  Subproblems 

Hearn  and  Lawphongpanich  (1989a)  viewed  the  difference  of  two 
successive  iterates  of  the  CP  algorithm  as  a  direction,  i.e.,  d  =  They 

showed  that,  when  Uu)  is  differentiable  at  d  is  an  ascent  direction. 
Otherwise,  the  nondifferentiability  at  implies  that  the  subproblem  at  the 
iteration  has  multiple  solutions  and  the  ascent  property  would  depend 
on  the  choice  of  In  any  case,  this  observation  suggests  the  inclusion  of  a 
line  search  step.  Below  is  one  version  of  the  CP  algorithm  with  line  search. 
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The  Cutting  Plane  Algorithm  with  Line  Search  (CPLS) 

Step  0:  Find  a  point  x°  e  X.  Let  v°  =  0  and  ^  =  1.  Go  to  Step  1. 

Step  1:  same  as  before. 

Step  2:  same  as  before. 

Step  3:  If  ),  then  ii’^)  solves  the  maximin  problem.:  Otherwise  (i.e., 

ry*  >  L(u*^)),  set 

(i)  If  fc  =  1;  set  and  ^  =  /c+l.  Go  to  Step  1. 

(ii)  Otherwise  {k  >  1),  let 

tmax  =  arg  max  +  td^y.O  <  t  <  tup) 

where  tup  is  the  maximum  value  of  t  for  which  remains 

feasible  to  U.  If  /max  ^  1/  then  pick  any  nonzero  €  [/max/I]- 
Otherwise,  let  /*^  G  [l//max]-:  Set  +  ti^d^  and  solve  the 

(sub)problem 

/  =  argmin{/(A:)  + 

X  €X 

L'  y*  is  not  unique,  select  y*^  €  so  that 

/■(y*)  *  "*s(y‘)  i/(y*)  +  '’‘x(y*)  ■  ^K)  (<) 

Set  and  k  =  k+l.-  Go  to  Step  1. 

Note  that  CPLS  produces  two  sets  of  dual  iterates,  u'^  and  where 
denotes  the  solution  of  the  master  problem  and  y*  is  a  point  along  the 
direction  d^  =  u^-  y'-'h;  Moreover,  the  cuts  are  generated  at  y*  instead  of  u^. 

In  Step  3(ii),  the  choice  of  step  length,  /^  in  the  direction  d^  is  inexact  to 
allow  for  inaccuracy  in  the  line  search  and  for  heuristic  selection  of  a  new  cut. 
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However,  setting  t'‘  =  t^ax  places  at  the  maximum  point  and  =  1  places 
at  M*  which  reduces  CPLS  to  the  basic  CP  algorithm. 

For  any  allowable  choice  of  the  resulting  is  feasible  to  the  master 
problem  and  we  have  the  following  relationship 
<  L(v^)  <  min(  f(x')  +  v^g(x^) ;  i  = 

If  L(v^)  =  w^,  then  CPLS  would  discover  that  {x^,  v^)  is  optimal  in  the  next 
iteration.  Otherwise  (i.e.,  L{v^)  <  w^),  adding  the  cut 
w  <  f(x^}  +  ug(x^) 

makes  the  point  {xv^,u^)  infeasible  to  the  next  master  problem  thereby  ensuring 
that  the  sequence  decreases  monotonically.- 

To  establish  convergence  for  CPLS,  it  is  assumed  that  either  U  is  compact 
or  L(u)  satisfies  the  following  condition 

[A]:  lim  L{u  +  td)  * 

t  —  oo 

for  all  u  €  U  and  for  all  of  its  direction  of  recession,  d  *  0  (see  page  61  of 
Rockafellar,  1970).  Among  the  three  instances  of  maximin  problems,  U  is 

compact  for  VI  problems  because  U  =  S  and  S  is  compact  by  the  standing 
assumption  in  Section  2.  For  optimization  problems,  U  =  (k,  >  0  :  z  =  1, ,  m} 

implies  that  every  component  of  its  direction  of  recession  must  be 
nonnegative.  Then,  the  requirement  (in  Theorem  3)  that  there  exists  an  a:°  €  X 
such  that  g(x°)  <  0  further  implies  that  condition  [A]  holds.  To  verify  this, 
note  that  for  any  ii  e  U  and  any  direction  of  recession  d  *  0 

L(ii  +  td)  =  min  {fix)  +  (u  +  td)g(x)}<  /(x^)  +  ug(x^)  +  tdg(x°). 
xeX 

Since  d>  0  andg(x°)  <  0,  dg(x°)  <  0  and  taking  the  limit  as  t  approaches  gives 
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\im  i{u  +  td)  <  f{x°)  +  ugix”)  +  limtdgix°)=  -«>. 

^  »  OO  f  ^  00 

Under  the  assumption  that  either  U  is  compact  or  condition  [A'|  holds,  the 
value  for  in  Step  3  of  CPLS  must  be  bounded.  This  in  turn  implies  that  v 

is  also  bounded  whenever  u  is  bounded  because 

b/l  -  +  ^  +  ^  max{l,  ij„ax)+l 

Then,  the  convergence  of  CPLS  follows  from  the  lemma  below. 

Lemma  4:  If  X  is  compact  and  there  exists  an  index  set  K  ^  (1,  2,  ...)  such  that 
the  subsequence  {ii'^]keK  converges  to  M“then  there  exists  a  subset 
K' Q  K  such  that  converges  to  r"  and 

lim  *  w*  *  liv°°] 
k€K'  ^ 

Proof:  Using  the  same  argument  as  in  Lemma  2,  it  can  be  shown  that 

>  ry  *  V;  *  0,1, 2,”-  (5) 

and  f{x^ )  +  +  u'^g^x^'j  -  £.. 

Using  inequality  (4)  in  Step  3(ii),  the  right  expression  is  bounded  above  as 
follows.: 

or  L(t^)  +  e  +  ii^g^x^y 

From  (5),  we  have 

w*  +£>  L[v’^)+  £  ^  /(a:*)  +  w*g(Ar^)  ^/(a^^)  +  ti'^g^x^'j  >  zi’“  >  7U* 
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Since  {«*^}fceK  converges,  must  be  bounded  for  all  k  £  K.  It  follows  from 
the  preceding  discussion  that  v  for  all  fc  €  K  must  be  bounded  as  well.  Thus, 
there  must  exist  a  subset  K'  S  X  such  that  the  subsequence  converges 

to  V".  So,  taking  the  limit  of  the  above  inequalities  with  respect  to  k  €  X',  we 
have 


w*  +e  > 


>L 


limn 

UeK' 


+  e  ^  w  * 


Since  e  is  arbitrary,  w*  =  L(r>*‘).. 


□ 


Given  Lemma  4,  obtaining  the  x  component  for  the  maximin  solution 
corresponding  to  i'*”  and  establishing  the  convergence  of  the  three  instances 
are  the  same  as  in  the  basic  CP  algorithm. 

At  the  end  of  Step  3(ii),  originally  defined  in  Step  2  is  replaced  with  y*^.; 
So,  y^now  defines  a  new  cut  for  the  (k+iy^  master  problem.  Moreover,  is 
not  arbitrarily  selected  when  alternate  optima  exist.  In  fact.  Step  3  requires 
that  y*^  satisfies  inequality  (4)  and  the  theorem  below  ensures  that  such  y*^ 
exists. 


Theorem  5:  In  Step  3  of  CPLS,  there  exists  a  y*‘  which  solves 

L{v^  )  =  min[/(A:)  + 

X  CX 

and  satisfies 

/V)  +  ^/(/)  +  ) 

where  is  part  of  the  solution,  to  the  k*^  master  problem. 

Proof:  Assume  that  >  1,  Denote  Then,  =  P  + 

(1-/3)  u  for  some  /3  €  (0,1)  and  by  concavity  of  L(u) 
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Uv'')  >  p  )  +  (1-p )  L(u)  >  L(u). 

LetX(v  )  denote  the  (compact)  set  of  solutions  to  the  subproblem.  To  obtain  a 
contradiction,  assume  that 

f(x)  +  g(x)  >  f(x)  +  V  g(x)  for  all  x  e  X(v^) 

Then 

(u  -v^)g(x)>Q  for  all  X  € 

which  implies  that 

min  [Ui^  -v^ )  g(x) :  x  €  X(v^)}  >  0 

k  k  k 

Thus,  the  directional  derivative  of  L(v)  at  v  in  the  direction  (u  -  v  )  is 

k  k  k 

positive,  i.e.,  (u  -  y  )  is  an  ascent  direction.  However,  v  is  a  point  on  the  line 

connecting  to  ti^  and,  by  the  concavity  of  L(u),  moving  toward  w^must 

k  k 

decrease  its  value.  This  contradicts  the  statement  that  (u  ~v  )  is  an  ascent 
direction,  (The  case  for  <  1  is  proved  similarly.)  □ 

Figure  1  illustrates  cuts  which  satisfy  or  are  'aliov^  ed'  by  inequality  (4).  In 
this  case,  there  are  an  infinite  number  of  allowable  cuts  and  any  of  them 
would  make  CPLS  converges. 

As  in  Magnanti  and  Wong  (1981),  we  say  that  the  cut,  w  <f(x')+iig(x'), 
dominates  or  is  stronger  than  the  cut,  w  <  f(x")  +  ug(.v")  if 

fix') -^ugix')^  fix")  Uigix")  Vu€U 

with  a  strict  inequality  for  at  least  one  «.  We  call  a  cut  pareto  optimal  if  no 
cut  dominates  it.  Since  a  cut  is  determined  by  at  €  X,  we  also  say  that  x' 
dominates  x”  (or  x'  is  stronger  than  x")  if  the  associated  cut  is  stronger,  and  we 
say  that  x  is  pareto  optimal  if  the  corresponding  cut  is  pareto  optimal. 
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It  is  interesting  to  note  that  every  allowable  cut  in  Figure  1  is  pareto 
optimal.  However,  this  is  not  always  true.  In  general,  there  are  allowable  cuts 
which  are  dominated.  Theorem  6  shows  how  to  modify  Step  3(ii)  of  CPLS  to 
generate  a  pareto  optimal  cut. 


Figure  1.  Allowable  Cuts  for  Step  3  of  CPLS 
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Theorem  6:  Let  X(y*^)  denote  the  solution  set  of  the  following  (sub)problem. 


min{/(x)  +y^g(x)}. 

X  €X 

If  y*^  uniquely  solves 

min  {f(x)  +  w^g(x)}, 

X  exly'l 

then  is  pareto  optimal. 

Proof:  Assume  the  contrary  that  y^  is  not  pareto  optimal,  i.e.,  there  exists  an 
X  €  X  and  xty^  such  that 

fix)  +  u^x)  <  f[y^]  +  i/^(y^)  Vti  €  U.  (6) 

In  particular,  setting  u  =  in  (6)  we  obtain 

Al)+i>*^(A:)S/(y‘)  +  !’‘s(y‘)-  (7) 


Since  y^  €  X(u*’), 


-  fix)+v'^g{x). 


(8) 


Combining  (7)  and  (8)  produces  the  following  equality 

‘  f(x) v’^S(x) 

that  is,  X  €  X(u*^).,  However,  setting  u  =  u  in  (6)  gives 

fix)  +  u'‘g(x}<  fly’ll +  /gly^j.. 


Since  x  ty^  and  y*''  uniquely  solves  the  2nd  minimization  problem,  this  last 
inequality  is  a  contradiction.  □ 
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If  in  Theorem  6  is  in  the  relative  interior  of  U ,  the  uniqueness 
assumption  for  can  be  dropped  and  the  pareto  optimality  follows  from 
Theorem  1  in  Magnanti  and  Wong  (1981).:  The  following  theorem  shows  that 
y*=  in  Theorem  6  defines  an  allowable  cut.; 


Theorem  7:  Let  be  as  defined  in  Theorem  6.,  Then 

/(y*-)  +  </(y*^)  +  ) 

Proof:  Assum.e  the  contrary  that 

M  >/(]/*■)  + 

Using  the  definition  of  y^  we  have  that 

min  {/(y)  +  i/''^(y))  >  /■(/)+ 
l/€X 


min  {/(y)  +  i(''^(y))  -  /(y'")  -  v’^gly^)  >  0. 
yeX(i’^) 


Since /(y*)  +  v'^giy’')  =  L{v'')  =  f(y)  +  v^g(y)  for  ally  e  X(v*’), 

min  Ifiy)  +  ii^ g{y)  ■  f{y)  -  y''^(y)}  >  0. 

yeX(rA) 


mm 

i/eX|z>^J 


>0- 


The  expression  on  the  left  side  is  the  directional  derivative  of  Liv^)  in  the 
direction  {u^-v^),  and  the  inequality  implies  that  u^-v^  is  an  ascent  direction 
which  is  impossible  for  our  choice  of  step  size  To  see  this,  assume  that  € 
[fmax/1]-  This  implies  that  is  on  line  joining  and  ii^,  where  =  v^''^  + 
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fmax  maximizes  L(u)  in  the  direction  So,  moving  away  from 

toward  can  only  decrease  the  function  value,  i.e.,  u^-v'^  cannot  be  an  ascent 
direction  at  v^.  The  argument  is  the  same  for  f*  e  [l,fmax]-  □ 

Theorems  6  and  7  demonstrate  that  it  is  possible  to  select  an  allowable  cut 
which  is  also  pareto  optimal.  Magnanti  and  Wong  (1981)  showed  that  pareto 
optimal  cuts  can  accelerate  the  convergence  of  the  cutting  plane  algorithm, 
particularly  for  Benders  decomposition.  Selecting  an  allowable  cut  or  a 
pareto  optimal  would  require  considerable  effort  in  general.  Hearn  and 
Lawphongpanich  (1989a  &  b)  described  a  heuristic  method  for  selecting  an 
allowable  cut.  The  method  consists  of  the  following  two  rules: 

i)  If  fmax  >  1/ set  f*' =  tmax  -  and 

ii)  If  fmax  ^  1/  set  =  tmax  +  E 

where  e  is  a  small  positive  number.  Figure  2  illustrates  how  these  rules  select 
a  cut.  The  function  L{ii)  is  nondifferentiable  at  Vmax  and  t,nax  <  1  in  Figures 
2(a)  and  (b).  So,  letting  =  tmax  +  E  would  set  to  the  right  of  Vmax  by  an  6 
amount.  However,  at  the  function  L(u)  is  differentiable  and  the  the  cut 
generated  here  is  the  line  tangential  to  L(u)..  In  Figure  2(a),  this  e-purturbation 
rule  chooses  the  only  nondominated  cut.  In  2(b),  there  are  an  infinite  number 
of  nondominated  cuts  and  all  of  which  are  convex  combinations  of  the  two 
'extreme'  cuts.  In  this  case,  the  rule  selects  one  of  the  extreme  cuts. 
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Figure  2.  A  Heuristic  Section  of  Allowable  Cut 
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4.2  Difficult  Subproblems 


In  this  situation,  one  alternative  is  to  solve  the  subproblem  only  approxi¬ 
mately  while  ensuring  that  effective  cuts  are  still  being  generated.  In 
Lagrangian  relaxation  or  subgradient  optimization  (see,  e.g.,  Fisher,  1981  and 
Polyak,  1969),  the  optimal  objective  function  value  is  used  to  define  the  step 
length  calculation  which  guarantees  convergence.  For  the  CP  algorithm,  the 
maximin  value,  w*,  analogously  defines  how  close  the  approximation  need  to 
be  in  order  to  generate  an  effective  cut.  In  general,  the  value  w*  is  unknown  at 
the  start  of  the  algorithm  and  must  be  estimated  to  make  the  resulting 
algorithm  effective.  However,  w*  =  0  for  VI  problems  (see,  e.g.,  Zuhovickii  et 
al,  1969  and  Auslender  1976).  This  allows  an  approximate  subproblem 
solution  to  be  easily  obtained. 

Below,  we  state  a  version  of  the  basic  CP  algorithm  which  solve  the  sub¬ 
problem  approximately.;  The  name  'nontangential'  is  due  to  the  fact  that,  by 
approximately  solving  the  subproblem,  the  generated  cuts  are  not  necessarily 
tangential  to  L(u). 

The  Nontangential  Cutting  Plane  Algorithm  (NTCP) 

Step  0:  same  as  before.. 

Step  1:  same  as  before. 

Step  2:  If  =  w*,  then  there  exists  an  optimal  solution  {w^,u^)  such  that 
L(«*^  =  XV*.  Otherwise  (i.e.,  xv*  <  xv^),  select  x*  €  X  such  that 

/(X*)  +  W*^g(X*^)  <  XU*:  (9) 

Setk  =k+l  and  go  to  Step  1., 
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Note  that  in  Step  2  need  not  be  an  optimal  solution  to  the  subproblem.  In 
fact,  the  (subproblem)  objective  function  value  at  only  need  to  be 
sufficiently  small,  i.e.,  no  larger  than  w*.  This  insures  that  is  infeasible 

to  the  master  problem  because 

f{x^)  +  ii^gix^)  £w*  <  w^. 

From  this  inequality,  it  can  be  concluded  that  is  still  a  monotonically 
decreasing  sequence.  However,  because  x^  does  not  necessarily  solve  the  sub¬ 
problem,  the  cut:  fix^)  +  ii^six^)  is  not  necessarily  tangent  to  L{ii)  at  (see 
Figure  3).  Nevertheless,  the  following  results  show  that  NTCP  can  still 
converges  to  the  desired  solution. 
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Theorem  8:  =  w*,  then  there  exists  an  optimal  solution  the  the  master 

problem,  such  that  L(u^)  =  w*.  That  is,  solves  the 

maximin  problem,  where 

=  min(/'(Ar)+M'^^(x)}. 

X  C  X 

Proof:  Assume  that  the  theorem  is  not  true.  Then,  for  every  pair  {w^,u^)  which 
is  optimal  to  the  master  problem,  the  following  must  hold 

<  min|/(x')  +  w^g(x'):/  -  0,...,{k  ■  1)|  *  •=  w*.  (10) 

However,  for  any  pair  {w,ii)  feasible,  but  nonoptimal,  to  the  master  prob¬ 
lem,  the  following  also  holds 

Liti)  ^  min|/(.r')  +  it^x‘):i  -  0,...  ,{k  - 1)}  ^  w  <  -  w\  (11) 

Combining  (10)  and  (11),  we  get 

L{u)<w*  for  all  «  €  L/ 

or  maxL(i/)<w*. 

u  eU 

This  is  a  contradiction  since  7t>*  is  the  maximin  value.  □ 

Lemma  9:  Assume  that  NTCP  generates  an  infinite  sequence  If  X  is  com¬ 
pact  and  there  exists  a  subset  X  £  (1,  2, ...  )  such  that  the  subse¬ 
quence  (»*‘)jtex  is  convergent,  say  to  the  limit  point,  then 

h 

w*  “  limta 
UK 

Proof:  Using  the  same  argument  as  in  Lemma  2,  it  can  be  shown  that  for  k 
sufficiently  large  and  k  e  K 


26 


(12) 


ZV*  <  W"  <  f{x^)  +  ll'”g{x^)  < f{x^)  +  u'^gix'^)  +  £. 
However,  by  the  choice  of  x^  in  Step  2  of  NTCP 

f{x^)  +  u^g{x^)  ^zv* 

Combining  (12)  and  (13)  yields  the  following 

zv*  ^  zv’^f^x^)  +  u“’^(x*)  ^/(x*^)  +  u'^gix'^)  e^zv*  +  z. 
Since  e  is  arbitrary,  we  can  conclude  that 

jU 

zo*  ”  limw 
k€K 


To  address  the  convergence  of  (t/  )j(.  to  where 

w*  -min{/(x)+u*^(x)}, 
r€X 


define  the  following 

i)  X"  = 

ii)  [x~|  -  the  closure  of  X“ 


Hi)  l!^{u)  »  min  ,{/(x')  + Mg(x')} 

/e{0,l . (jt-i)}^  ^ 

iv)  C{u)  =  min  {fix) +  ugix)} 

.vclx^l 


From  (iii)  and  (iv),  it  is  clear  that 


L”(i<)<L*'*\m)<L^m)  VueU, 
and  it  can  be  shown  that 

lim  L^i<)  =  L”(t<)  Vw€U. 

jt-*  00 


(13) 

(14) 


□ 
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k 

In  other  words,  [L(u  is  a  monotonic  sequence  of  functions  which  converges 

pointwise  to  Furthermore,  note  that 

k  k 

u  =  argmax  L  («). 

«  eU 

k  ^ 

If  the  subsequence  {«  converges  to  u  ,  then  it  follows  frofn  Theorem  3.7  of 
Wets  (1983)  [see  also  the  results  in  Wets,  1980]  that 

M*  “  are  max  L”(m), 

M  €U 

and  (14)  also  implies  that 

w*  =  L  (u  )  «  hm  w^. 

kcK 

Lemma  10:  Under  the  same  assumptions  as  in  Lemma  9,  u*  solves  the 
following  problem: 

max  L* (u). 
ueU 

Proof:  Note  that 

w*  *  min  {/(x)  + u*  g(x)}  <  min  {f(x)  +  u*  g(x)} 

$  min  {f{x^)  +  u'*  g{x')] 

where  the  first  inequality  follows  from  the  fact  that  [X”]  is  a  subset  of  X,  the 
second  from  the  fact  that  (0,1,2,.. .,(it-I)}  is  a  subset  of  [X"],  and  the  last  from  the 

th 

fact  that  u*  does  not  necessarily  solve  the  k  master  problem. 

The  above  sequence  of  inequalities  can  be  summarized  as  follows 

w*<  C°{u*)  <  w^... 

By  taking  the  limit  as  k  ,k  £  K  ,  and  invoking  Lemma  9,  we  have  that 
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L”(t<*)  =  w*=  max  L“(u) 
u  €U 

and  the  proof  is  complete. □ 

An  immediate  consequence  of  Lemma  10  is  that  if  the  problem 

max  lT (u) 
ueU 

admits  a  unique  solution,  then  {m  must  converge  to  u*.  Given  Lemma  9  and 
10,  the  convergence  of  NTCP  for  the  three  instances  of  maximin  problem  can 
be  established  as  in  the  case  of  the  basic  CP  algorithm  except  for  Benders 
decomposition.  To  insure  that  the  NTCP  algorithm  for  Benders 
decomposition  converges  finitely,  the  x*  chosen  in  Step  2  of  NTCP  must  also 
be  an  extreme  point  of  the  region  X. 

It  is  interesting  to  specialize  NTCP  to  VI  problems.  Because  of  the 
structure  of  Vis,  the  task  of  selecting  an  in  Step  2  is  much  easier  and  the 
stopping  rule  and  the  convergence  result  can  both  be  strengthened.  Recall 
that  the  maximin  problem  for  Vis  takes  the  following  form 

to*  =  max  min  {F{x)x  •  F(x)«} 
ueS  xeS 

where  S  is  nonempty,  convex  and  compact  subset  of  R"  and  F(x)  is  a  vector 
valued  function  mapping  R”  into  R".  Moreover,  F(x)  is  further  assumed  to  be 
strongly  monotone  which  implies  that  there  is  a  unique  solution,  u*,  to  the  VI 
problem,  i.e., 

F{u*){x  •  u*)  >0  V  X  €  S. 

Below,  we  state  the  version  of  NTCP  algorithm  for  Vis. 
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The  Cutting  Plane  Algorithm  for  Variational  Inequalities  (CPVI) 

Step  0:  Leta:°  €  S.  Setk  =  l  and  go  to  Step  1. 

Step  1:  Solve  the  (k^)  master  problem 

max  w 

s.t.  lu  ^  F(x')x^  F{x')u  i  =  0, {k-1) 

u  £  S 

Let  be  an  optimal  solution  and  go  to  Step  2. 

Step  2:  If  =  0,  stop  and  x'  for  some  i  e  (0, (^-1)}  is  a  solution..  Otherwise, 
select  a:*'  €  S  such  that 

^  0. 

Set  k  =  ^+1  and  go  to  Step  1. 

The  stopping  rule  in  Step  2  follows  from  the  fact  that  w*  =  0  for  VI 
problems  (see,  e.g.,  Zuhovickii  et  al.  1969  and  Auslender  1976).  Theorem  11 
and  12  below  describe  how  to  obtain  the  solution  to  the  VI  when  CPVI 
terminates  finitely  and  when  it  generates  an  infinite  sequence,  respectively. 

Theorem  11:  If  =  0  and  F{x^){x'-it^)  =  0,  then  ^  solves  the  VI  problem.; 

Proof:  Under  the  strong  monotonicity  assumption,  the  solution  to  the  VI 
problem  must  be  unique  (see,  e.g.,  Auslender  (1976)  and  Hearn  et  al.  (1984)]. 
Thus,  there  must  be  only  one  such  that 

0  =  =  F{x’)  {y!  -M*=) 

and  for  i  ^  / 

Q  =  <  Fix')  {x‘  -  !<*■'). 
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Then,  the  master  problem  can  be  reduced  to  a  problem  with  only  one  cut, 
i.e.. 


Q  =  w^  =  max  w 

s.t.  w  <  F{x')  {x’-u) 

ueS, 

or  0  =  “  max  pfx^  jfx^  ' m). 

ueS  ^  ' 


However, 

0  =  max f(x^)[x^  -  «)  >  f(x^)(x^  - 1/)  VweS 

Multiply  through  by  -1  to  obtain 

f[x^)[i(  -  x^)  >  0  VueS., 

That  is,  xi  solves  the  VI  problem.  ^ 


Theorem  12:  The  solution  to  the  VI  problem,  u*,  is  a  limit  point  of  the  set  [  X”]. 

th 

Proof:  For  each  k,  let  i(k)  denote  an  index  of  an  active  cut  for  the  k  master 
problem,  i.e.,  i(k)  satisfies 


Then,  we  have 


a 


riik)  . 


{Fix^^^)  ■  F{u*)){x‘^^'>  •  u*) 

Flx'^^Xx^'^^  -  u*)  -  F{in{x‘^’^'>  •  in 
F(x»(M)(x'U)  .  „*) 


The  first  inequality  follows  from  the  definition  of  strong  monotonicity  and  the 
second  from  the  fact  that  u*  solv  es  the  VI  problem,  i.e.. 
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F{it){x  •  u*)  >  0 


Yx  eS. 


Since  S  is  compact,  there  must  exist  a  subset  K  ^  {0,  1,  2,.  .  .  }  satisfying  the 
condition  in  Lemma  9,  thereby  having  the  property  that 

0  =  lim  w 
k£K 

Thus,  from  the  above  sequence  of  inequalities,  it  must  be  true  that 

lim  I  -  w  *  ^  -  0  or  lim  -  u\ 
k£K'  keK 

i(k) 

From  the  above  theorem,  there  exists  a  subsequence  of  active  cuts,  {x  } , 
converging  to  u*.  Alternately,  Lemma  10  also  implies  that  the  sequence  {u  } 
also  converges  to  u*  when  it  'strongly'  solves  the  VI  problem  (see  Figure  4), 
i.e., 

F(i'  )(x  -  II*)  >0  V  X  eS  &  X  ^  u*. 

In  this  case,  the  problem:  max(L”(i<>) :  «  e  S  }  admits  a  unique  solution.  To 
verify  this,  assume  that  ii'  *  u*  is  an  alternate  solution  to  the  problem.  Then 

L”(w’)“  min  {F(x)(x-t<')}^F(M*X«*  •m')<0, 

,£|x-| 

where  the  first  inequality  follows  from  the  fact  that  u*  €  [  X”]  and  ihe  second 
from  the  fact  that  u*  'strongly'  solves  the  VI  problem.  However,  from  Lemma 
10,  u*  also  solves  max{L°°(u)  :.ii€  S  }  and  L°"(u*)  =w*  =  0.  Thus,  the  above 
inequalities  is  a  contradiction  since  L°°(u*)  =  L”fw';  =  0.  Therefore,  when  u* 
'strongly'  solves  the  VI  problem,  the  entire  sequence  {  u  }  must  converge  to  u*, 

k 

since  every  convergent  subsequence  of  {u  }  must  converge  to  a  solution  of  the 
problem  ma\[L°“(u) :  u  €  S  }  which  has  a  unique  solution  (see,  page  234  of 
Bazaraa  and  Shetty,  1979), 
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Figure  4:  A  'Strong'  Solution  to  a  Variational  Inequality  Problem 


The  rate  of  convergence  of  CPVI  depends  in  part  on  the  choice  for  in 
Step  2.  The  choice  requiring  the  least  effort  is  due  to  Zuhovickii  et  al.  (1979); 
they  set  x*’’  =  «*•'.  Under  this  choice 

•  i/')  »0. 

Nguyen  and  Dupuis  (1984)  proposed  another  choice  of  xK  They  let  x'^  be  the 
solution  to  the  VI  problem  over  the  line  segment  joining  x'^~^  and  denoted 
as  t[x^~^ ,  Thus,  X*’  e  ,  i/^]  and  satisfies 

Since  w*  €  f’[x*’“^ , 
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>  0  or  •  u^j  <  0. 

So,  Nguyen  and  Dupuis  choice  of  satisfies  the  requirement  in  Step  2. 
Figure  5  illustrates  this  choice  of  x^.  In  Figure  5(a),  if  F(x*^)  is  a  gradient  of  a 
function,  say, /(x),  then  x'^  would  be  the  unconstrained  minimizer  of  a  one- 
dinicnsional  optimization  problem.  In  Figure  5(b),  the  'unconstrained 
minimizer'  is  to  the  left  of  So,  x  must  be  set  equal  to  However,  this 
suggests  a  generalization  of  the  choice  for  x^  by  Nguyen  &  Dupuis. 

Let 

a^,p  “  max|a:a  >  0  andx*^'^  +  a(u*^  -  eS} 

and  set 

t\,p  “X^'^  +0!up(w*' 

Then,  choose  x*^  €  i^up  /  1  such  that 

Flx^'Ky -x'')>0  Yy  £e[v^p,x’'''^\. 

Since  e  i[  Uup ,  ], 

F(x'')(i/*'  -  r^')  >0  or  f(x'^)(x^  •  <  0., 

Thus,  the  new  choice  of  x*'  also  satisfies  the  requirement  in  Step  2.  Figure  6 
illustrates  the  new  choice  of  x*’'. 


34 


Figure  5,  Nguyen  &  Dupuis's  Choice  for 
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5.  CONCLUSION 


This  paper  addresses  the  cutting  plane  algorithm  which  has  been  pro¬ 
posed  for  problems  derived  from  two  different  but  related  areas  in  operations 
research:  optimization  and  equilibrium  problems.  In  optimization,  the  cut¬ 
ting  plane  algorithm  is  often  used  to  solve  the  master  problem  from  Benders 
decomposition  or  Lagrangian  dual  of  a  nonlinear  program.:  For  the  equi¬ 
librium  problem,  the  algorithm  has  been  applied  only  to  finite  dimensional 
variational  inequalities.  To  unify  ideas  in  the  two  areas,  this  paper  views 
problems  in  both  areas  as  maximin  problems.  This  establishes  a  common 
framework  for  analyzing  and  examining  convergence  properties  of  various 
schemes  for  enhancing  the  cutting  plane  algorithm.  In  particular,  the  analysis 
also  leads  to  several  interesting  new  results.  First  is  the  relationship  between 
adding  a  line  search  step  and  generating  'deep'  or  pareto  optimal  cuts. 
Second  are  the  concept  of  generating  nontangential  cuts  and  the  justification 
for  solving  the  subproblem  approximately.  The  last  are  an  alternate  proof  of 
convergence  and  a  generalization  of  an  existing  method  for  generating  cuts  in 
the  case  of  variational  inequalities.. 

It  has  been  well  demonstrated  that  the  acceleration  strategies  discussed 
herein  actually  reduced  the  computing  time.  Hearn  and  Lawphongpanich 
(1989a  &  b)  compared  the  CP  algorithm  against  CPLS  on  25  Linear  and  25 
quadratic  integer  programming  problems.  The  number  of  variables  in  these 
problem  ranges  from  20  to  100  with  number  of  constraints  ranges  from  10  to 
50.  They  concluded  that  the  addition  of  line  search  saves  cpu  time  on  the 
average  40%  for  linear  problems  and  70%  for  quadratic.  The  rather  large 
saving  for  the  quadratic  case  is  due  to  the  fact  that  the  line  search  only  require 
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two  dual  function  evaluations.  In  Hearn  and  Lawphongpanich  (1990),  the 
same  comparison  was  made  on  a  traffic  assignment  problem  from  Goffin 
(1987)  with  22  arcs,  14  nodes,  and  23  origin-destination  pairs.  The  saving  due 
to  line  search  is  approximately  50%. 

As  for  variational  inequalities,  its  subproblem  is  generally  a  nonlinear 

program.  Thus,  by  entirely  eliminating  the  subproblem  from  the  steps  of  the 

CP  algorithm,  CPVI  can  only  accelerate  the  basic  algorithm.  To  demonstrate 

that  CPVI  is  competitive  with  existing  methods  for  variational  inequalities, 

Nguyen  and  Dupuis  (1984)  compared  CPVI  with  =  v*  against  the  well- 

regarded  Frank-Wolfe  algorithm  (see,  Frank  and  Wolfe,  1956  and  LeBlanc  et 

al.,  1975)  for  the  traffic  assignment  problem.  They  showed  that  CPVI  with  x  = 

V*  compares  favorably  with  Frank-Wolfe  on  problems  with  19  arcs,  11  nodes 

and  4  origin-destination  pairs  to  problems  with  2836  arcs,  1052  nodes  and  147 

origin-destination  pairs..  Also  shown  is  the  fact  that  CPVI  with  x^  =  v*  is 

kk  k 

superior  to  the  one  with  x  =  u  This  is  not  unexpected  since  setting  x  =v*  is 

k  k-1 

tantamount  to  performing  a  line  search  along  the  direction  ii  -  x  (see 
Hammond,  1984). 
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